Skip to content

Fix memory tracking of ES|QL sample agg#128467

Merged
jan-elastic merged 3 commits intoelastic:mainfrom
jan-elastic:fix-memory-esql-sample
May 27, 2025
Merged

Fix memory tracking of ES|QL sample agg#128467
jan-elastic merged 3 commits intoelastic:mainfrom
jan-elastic:fix-memory-esql-sample

Conversation

@jan-elastic
Copy link
Contributor

fixes #128024

@elasticsearchmachine elasticsearchmachine added needs:triage Requires assignment of a team area label v9.1.0 labels May 26, 2025
@jan-elastic jan-elastic requested a review from ivancea May 26, 2025 14:28
@jan-elastic jan-elastic added >bug >test-failure Triaged test failures from CI :ml Machine learning Team:ML Meta label for the ML team labels May 26, 2025
@elasticsearchmachine elasticsearchmachine added needs:risk Requires assignment of a risk label (low, medium, blocker) and removed needs:triage Requires assignment of a team area label labels May 26, 2025
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/ml-core (Team:ML)

Copy link
Contributor

@ivancea ivancea left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! LGTM!

Comment on lines 99 to 101
BytesRefBlock bytesRefBlock = (BytesRefBlock) block;
try ($Type$Block.Builder $type$Block = driverContext.blockFactory().new$Type$BlockBuilder(bytesRefBlock.getPositionCount())) {
try (
block;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: What about moving the cast inside the try? So we avoid this extra block;. Not much of an improvement, but I think it makes the intent a bit clearer ("Now we'll consume this block, which is a BytesRefBlock"), and we use both variables from here on

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SGTM

Maybe it's slightly obscure that handles closing the original block.

Anyway, it also conveniently solves my spotless/autogen issue (see the autocommit). Depending the the strlen of the type (Int or BytesRef), this block should go on 1 or 2 lines right now.

@jan-elastic jan-elastic merged commit e352d2c into elastic:main May 27, 2025
19 checks passed
@jan-elastic jan-elastic deleted the fix-memory-esql-sample branch June 3, 2025 08:00
jan-elastic added a commit to jan-elastic/elasticsearch that referenced this pull request Jun 18, 2025
* Fix memory tracking of ES|QL sample agg

* [CI] Auto commit changes from spotless

* polish code

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
jan-elastic added a commit to jan-elastic/elasticsearch that referenced this pull request Jun 18, 2025
* Fix memory tracking of ES|QL sample agg

* [CI] Auto commit changes from spotless

* polish code

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
jan-elastic added a commit to jan-elastic/elasticsearch that referenced this pull request Jun 18, 2025
* Fix memory tracking of ES|QL sample agg

* [CI] Auto commit changes from spotless

* polish code

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
jan-elastic added a commit to jan-elastic/elasticsearch that referenced this pull request Jun 19, 2025
* Fix memory tracking of ES|QL sample agg

* [CI] Auto commit changes from spotless

* polish code

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
jan-elastic added a commit that referenced this pull request Jun 19, 2025
* ES|QL SAMPLE aggregation function (#127629)

* ES|QL SAMPLE aggregation function

* [CI] Auto commit changes from spotless

* ThreadLocalRandom -> SplittableRandom

* Update docs/changelog/127629.yaml

* fix yaml test

* Add SampleTests

* docs + example

* polish code

* mark generated imports

* comment with algorith description

* use Randomness.get()

* close properly

* type checks

* reuse hash

* regen some files

* [CI] Auto commit changes from spotless

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>

* Fix + unmute SampleTests (#127959)

* Fix memory tracking of ES|QL sample agg (#128467)

* Fix memory tracking of ES|QL sample agg

* [CI] Auto commit changes from spotless

* polish code

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>

* ESQL: Unclean generated imports (#127723)

This removes a ton of the tricky juggling we do for generated java files
to keep the imports in order. Instead, we just live with them being out
of order a little. It's not great, but it's so so so much easier than
the terrible juggling we had been doing.

* ESQL: Disable format checks on generated imports (#127648)

This builds the infrastructure to disable spotless and some checkstyle
rules on generated imports. This works around the most frustrating part
of ESQL's string template generated files - the imports. It allows
unused and out of order imports. This can let us remove a lot of
cumbersome, tricky, and fairly useless `$if$` blocks from the templates.

---------

Co-authored-by: elasticsearchmachine <infra-root+elasticsearchmachine@elastic.co>
Co-authored-by: Nik Everett <nik9000@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

>bug :ml Machine learning needs:risk Requires assignment of a risk label (low, medium, blocker) Team:ML Meta label for the ML team >test-failure Triaged test failures from CI v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[CI] SampleDoubleAggregatorFunctionTests testSimpleWithCranky failing

3 participants